Other-Settings Generalizability in IS Research

This paper presents a simple conceptualization of generalization, called other-settings generalization, that is valid for any IS researcher who claims that his or her results have applicability beyond the sample where data were collected. An other-settings generalization is the researcher’s act of arguing, based on the representativeness of the sample, that there is a reasonable expectation that a knowledge claim already believed to be true in one or more settings is also true in other clearly defined settings. Features associated with this conceptualization of generalization include (a) recognition that all human knowledge is bounded, (b) recognition that all knowledge claims—including generalizations—are subject to revision, (c) an ontological assumption that objective reality exists, (d) a scientific-realist definition of truth, and (e) identification of the following three essential characteristics of sound other-settings generalizations: (1) the researcher must clearly define the larger set of things to which the generalization applies; (2) the justification for making other-settings generalizations ultimately depends on the representativeness of the sample, not statistical inference; (3) representativeness is judged by comparing key characteristics of the proposition being generalized in the sample and target population. The paper concludes with the recommendation that future empirical IS research should include an explicit discussion of the other-settings generalizability of research findings.